Papers with variational auto-encoder
Deep Bayesian Natural Language Processing (P19-4)
Copied to clipboard
| Challenge: | Introduction to deep Bayesian learning for natural language addresses the fundamentals of statistical models and neural networks. |
| Approach: | This tutorial addresses the advances in deep Bayesian learning for natural language . it focuses on advanced Bayessian models and deep models . authors present case studies and domain applications to tackle different issues . |
| Outcome: | This tutorial focuses on advanced Bayesian models and deep models for natural language . case studies and domain applications are presented to tackle different issues in deep Bayessian processing, learning and understanding. |
Topic-Guided Variational Auto-Encoder for Text Generation (N19-1)
Copied to clipboard
Wenlin Wang, Zhe Gan, Hongteng Xu, Ruiyi Zhang, Guoyin Wang, Dinghan Shen, Changyou Chen, Lawrence Carin
| Challenge: | Experimental results show that our model outperforms its competitors on both unconditional and conditional text generation. |
| Approach: | They propose a topic-guided variational auto-encoder model for text generation that specifies a Gaussian mixture model and a neural topic module to generate sentences under the topic. |
| Outcome: | The proposed model outperforms existing variational auto-encoders on unconditional and conditional text generation, and can generate semantically-meaningful sentences with various topics. |
Document Hashing with Multi-Grained Prototype-Induced Hierarchical Generative Model (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Existing document hashing methods only consider flat semantics of documents, preserving hierarchical semantics. |
| Approach: | They propose a hierarchical generative model that can model and leverage hierarchic semantics . they introduce hierarchically-based prototypes into the model to construct a Hierarchical prior distribution . |
| Outcome: | The proposed model outperforms baseline methods on hierarchical and flat datasets. |
RegaVAE: A Retrieval-Augmented Gaussian Mixture Variational Auto-Encoder for Language Modeling (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing research on retrieval-augmented language models has two main problems: determining what information to retrieve and effectively combining retrieved information during generation. |
| Approach: | They propose a retrieval-augmented language model that captures current and future information from source and target text into a latent space. |
| Outcome: | The proposed model is more efficient than explicit raw text, but limited by context length and noise. |
Enhancing Dual-Encoders with Question and Answer Cross-Embeddings for Answer Retrieval (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Existing approaches to solve question answering (QA) problems are limited by the need for text generation and answer retrieval. |
| Approach: | They propose to introduce QA interaction features in scoring function but at the cost of low efficiency in inference stage. |
| Outcome: | The proposed framework significantly outperforms the state-of-the-art method on multiple answer retrieval datasets. |
Towards Reinterpreting Neural Topic Models via Composite Activations (2022.emnlp-main)
Copied to clipboard
| Challenge: | Most Neural Topic Models (NTMs) use a variational auto-encoder framework producing K topics limited to the size of the encoder’s output. |
| Approach: | They propose a model-free two-stage process to reinterpret NTM and derive further insights on the state of the trained model. |
| Outcome: | The proposed model-free process decouples the strict interpretation of topics from the original NTM and evaluates them on a large external corpus. |
Implicit Deep Latent Variable Models for Text Generation (D19-1)
Copied to clipboard
| Challenge: | Variational auto-encoders have been used for text generation but their representation power is limited due to two reasons. |
| Approach: | They advocate sample-based representations of variational distributions for natural language . they further develop an LVM to directly match the aggregated posterior to the prior . |
| Outcome: | The proposed model can be viewed as a natural extension of VAEs with a regularization of maximizing mutual information, mitigating the "posterior collapse" issue. |
Document Hashing with Mixture-Prior Generative Models (D19-1)
Copied to clipboard
| Challenge: | Existing generative hashing methods only consider the use of simple priors, which limits them to further improve their performance. |
| Approach: | They propose to use Gaussian and Bernoulli priors to generate hashing codes . they propose to cast a Gausssian latent representation into binary code . |
| Outcome: | The proposed models outperform existing methods on a benchmark dataset using Gaussian and Bernoulli priors. |
Effective Estimation of Deep Generative Language Models (2020.acl-main)
Copied to clipboard
| Challenge: | Existing techniques for parameterisation of probabilistic models by deep neural networks are difficult to use in language modelling due to posterior collapse. |
| Approach: | They propose to use variational auto-encoder to estimate probabilistic models of language by deep neural networks. |
| Outcome: | The proposed model performs reasonably well given enough resources, but a favourite can be named based on convenience. |